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Foreword 

The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (Public Law 
104-193) replaced the Aid to Families With Dependent Children (AFDC) program with the 
Temporary Assistance to Needy Families (TANF) program. Under TANF, welfare recipients 
are required to work, and States are responsible for designing and operating their own 
welfare-to-work plans. However, if welfare reform is to succeed, States must develop 
policies and programs to address the effects of substance abuse on the employability of 
TANF recipients. 

Many States allow the use of TANF funds to support substance abuse treatment for 
recipients as part of work-readiness efforts. To ensure implementation of sound welfare-to- 
work policies and programs and to justify allocation of TANF resources to substance abuse 
services, State officials need reliable and timely information on the effects of substance 
abuse treatment on employment outcomes for welfare recipients. 

This document responds to this information need by analyzing the effects of substance 
abuse treatment on employment outcomes among AFDC recipients in Washington State. It 
presents findings regarding the general patterns of employment and earnings for 
Washington State AFDC recipients before and after substance abuse treatment; the extent 
of welfare dependence before and after treatment; the impact of different substance abuse 
treatment modalities on employment outcomes; and whether the amount of time on welfare 
before being treated for substance abuse affects employment outcomes. 

The document presents an analysis of the pattern of employment and earnings before and 
after the implementation of TANF. The document also discusses the value of developing the 
capacity to use State computer databases for evaluating clients and tracking client 
outcomes. 



The Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health 
Services Administration (SAMHSA), are pleased to disseminate this document to State 
substance abuse agencies, State welfare agencies, State health planning authorities, State 
advisory committees, providers, consumers, and other stakeholders. The document should 
serve as a tool as stakeholders confront the ramifications of welfare reform for substance 
abuse treatment. 
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Acting Administrator 
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A. Introduction 

Questions about the effects of substance abuse treatment have recently been given new urgency 
by welfare reforms enacted by Congress in the last 4 years. In 1996, Congress enacted the 
Personal Responsibility and Work Opportunity Reconciliation Act (Public Law 104-193) in an 
effort to move people off welfare and to establish strong incentives to work. The new law 
replaced the Aid to Families With Dependent Children (AFDC) program with Federal block 
grants to States to provide temporary assistance to needy families (TANF); it also imposed a 5- 
year lifetime limit on welfare payments to families with a minor child.* A key assumption 
underlying the 5-year lifetime limit is that most welfare recipients can obtain jobs that enable 
them to move from welfare dependency to economic self-sufficiency. 

Obtaining a job and achieving economic self-sufficiency is especially difficult for welfare 
recipients with substance abuse problems. At a minimum, they need treatment for their substance 
abuse problems. Substance abuse treatment works, but little is known about the effects of 
substance abuse treatment on employment outcomes for welfare recipients with substance abuse 
problems. 

The primary purpose of this study was to assess the effect of substance abuse treatment on 
employment outcomes and earnings among AFDC clients admitted to treatment in Washington 
State. A second purpose was to illustrate the potential for States to use data from State 
administrative databases for tracking client outcomes and conducting timely evaluation research. 
The first chapter of this report discusses the context and purpose of the study. The second 
chapter outlines the methods, data, and analytical procedures. The third chapter presents the 






study results, including findings from statistical analyses of the effects of substance abuse 
treatment on employment outcomes among 5,664 Washington State AFDC clients admitted to 
substance abuse treatment. The report closes with a chapter on conclusions and implications. A 
technical appendix presents additional details on the statistical analyses. (Refer Chapter I for 
details) 

B. Methods, Data, and Analytical Procedures 

The study sample consisted of 5,664 individuals admitted to substance abuse treatment in 
Washington State from July 1994 to June 1996 who were receiving financial assistance from 
AFDC at the time of their admission. Data on these individuals' treatment and employment 

* States may exempt up to 20 percent of their welfare caseload from the 5-year lifetime limit, 
outcomes-employment, earnings, and welfare payments-were obtained from administrative 
databases maintained by the State of Washington. 

Employment data for the study spanned 5 years, and welfare payment data spanned 4 years, both 
beginning in July 1993. 

The evaluation of the effects of substance abuse treatment on employment outcomes focused on 
three treatment modalities (see Table ES-1). 



Table ES-1. Distribution of Washington State AFDC Clients 

Across the Three Substance Abuse Treatment Modalities 

Outpatient treatment: 


Treatment group—received 90 days or more of treatment (n = 870) 


Comparison group—received under 21 days of treatment (n = 1,131) 

Intensive inpatient (21- or 28-day non-hospital) treatment: 


Treatment group—completed treatment (n = 1,139) 


Comparison group—detoxification clients with no other treatment (n 
= 261) 

Methadone maintenance: 


Treatment group—received 120 days or more of treatment (n = 143) 


Comparison group—received under 60 days of treatment (n = 240) 


Analytic procedures included a descriptive analysis and a multivariate statistical analysis. Results 
from both analyses are summarized below. (Refer Chapter II for details) 



































C. Results 


In the descriptive analysis, the 5,664 Washington State AFDC clients' earnings and welfare 
payments were profiled for several quarters before and after substance abuse treatment. 

• Earnings levels among AFDC clients in the sample were very modest. In the year before 
substance abuse treatment, the clients' quarterly earnings declined. In the 2years after 
treatment, their quarterly earnings increased. 

• Welfare payments to the AFDC clients, in contrast to earnings, increased in the year- 
before the clients entered substance abuse treatment. In the IV 2 -yearperiod after- 
treatment, welfare payments declined. More than one-fourth (27 percent) of the AFDC 
clients in the sample remained on welfare throughout the 2 V 2 -year tracking period. 

Because descriptive information does not take into account differences in age, gender, education, 
or other factors that may affect treatment outcomes, conclusions about the effects of substance 
abuse treatment on AFDC clients' earnings and welfare payments should not be made on the 
basis of descriptive information alone. To adjust the data for those factors, statistical analyses 
were perfonned. 

Statistical analyses of the effects of substance abuse treatment on employment outcomes among 
the 5,664 AFDC clients in the sample demonstrated that substance abuse treatment had a 
significant positive effect on employment outcomes: 

• AFDC clients in the three substance abuse treatment groups were 50 to 100percent more 
likely (p < .05) to become employed in the 2-year post-treatment period than their 
counterparts in the comparison groups. 

• AFDC clients in the three substance abuse treatment groups had adjusted aggregate 
earned income levels in the 2-year post-treatment period that were approximately $1,700 
to $3,200 (p < .05) higher than the income levels among their counterparts in 
comparison groups. 

• AFDC clients who had been on welfare for more quarters earned substantially less than 
clients who had been on welfare for less time. Nevertheless, substance abuse treatment 
was associated with significant gains in earnings for all AFDC clients, regardless of the 
length of time they were on welfare. 

The finding that substance abuse treatment had positive effects on AFDC clients' employment 
and earnings was robust across all three of the treatment modalities analyzed: outpatient 
treatment, intensive inpatient treatment, and methadone maintenance (Refer Chapter III for 
details) 

D. Conclusions and Implications 

1. The Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes 



This study's finding that substance abuse treatment had positive effects on employment and 
earnings among AFDC clients in Washington State is notable, especially because the primary 
goal of substance abuse treatment is rehabilitation, not employment. It is important to emphasize, 
however, that despite gains in income following substance abuse treatment, earnings among the 
AFDC clients in the sample remained quite low-too low for most clients to achieve economic 
self-sufficiency. Unless these welfare clients receive vocational services in conjunction with 
substance abuse treatment, they will probably be unable to become economically self-sufficient. 

2. The Utility of Computerized State Administrative Databases for Tracking Client 
Outcomes and Performing Evaluation Research 

Few States have the resources or time to conduct studies involving large-scale primary data 
collection. Such studies may take years to complete and may produce information of diminished 
value given the rapid policy and programmatic changes taking place in many States. 

This study in Washington State illustrates a useful approach to performing evaluation research- 
that is, analyzing data provided by computerized State administrative databases. State databases 
contain client-level data representing important outcome indicators of substance abuse treatment- 
including employment, public assistance payments, Medicaid utilization and expenditures, and 
criminal justice information. By utilizing such information, States can greatly improve their 
capacity to perform timely evaluation and outcome studies. (Refer Chapter IV for details) 
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The primary purpose of this study was to assess the effect of substance abuse treatment on 
employment outcomes and earnings among AFDC clients admitted to treatment in Washington 
State. A second purpose was to illustrate the potential for States to use data from State 
administrative databases for tracking client outcomes and conducting timely evaluation research. 
The first chapter of this report discusses the context and purpose of the study. The second 
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Comparison group—received under 60 days of treatment (n = 240) 


Analytic procedures included a descriptive analysis and a multivariate statistical analysis. Results 
from both analyses are summarized below. (Refer Chapter II for details) 

C. Results 

In the descriptive analysis, the 5,664 Washington State AFDC clients' earnings and welfare 
payments were profiled for several quarters before and after substance abuse treatment. 

• Earnings levels among AFDC clients in the sample were very modest. In the year before 
substance abuse treatment, the clients' quarterly earnings declined. In the 2years after 
treatment, their quarterly earnings increased. 

• Welfare payments to the AFDC clients, in contrast to earnings, increased in the year 
before the clients entered substance abuse treatment. In the IV 2 -yearperiod after 
treatment, welfare payments declined. More than one-fourth (27 percent) of the AFDC 
clients in the sample remained on welfare throughout the 2 V 2 -year tracking period. 

Because descriptive information does not take into account differences in age, gender, education, 
or other factors that may affect treatment outcomes, conclusions about the effects of substance 
abuse treatment on AFDC clients' earnings and welfare payments should not be made on the 
basis of descriptive information alone. To adjust the data for those factors, statistical analyses 
were perfonned. 

Statistical analyses of the effects of substance abuse treatment on employment outcomes among 
the 5,664 AFDC clients in the sample demonstrated that substance abuse treatment had a 
significant positive effect on employment outcomes: 

• AFDC clients in the three substance abuse treatment groups were 50 to 100percent more 
likely (p < .05) to become employed in the 2-year post-treatment period than their 
counterparts in the comparison groups. 

• AFDC clients in the three substance abuse treatment groups had adjusted aggregate 
earned income levels in the 2-year post-treatment period that were approximately $1,700 
to $3,200 (p < .05) higher than the income levels among their counterparts in 
comparison groups. 

• AFDC clients who had been on welfare for more quarters earned substantially less than 
clients who had been on welfare for less time. Nevertheless, substance abuse treatment 
was associated with significant gains in earnings for all AFDC clients, regardless of the 
length of time they were on welfare. 

The finding that substance abuse treatment had positive effects on AFDC clients' employment 
and earnings was robust across all three of the treatment modalities analyzed: outpatient 










treatment, intensive inpatient treatment, and methadone maintenance (Refer Chapter III for 
details) 

D. Conclusions and Implications 

1. The Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes 

This study's finding that substance abuse treatment had positive effects on employment and 
earnings among AFDC clients in Washington State is notable, especially because the primary 
goal of substance abuse treatment is rehabilitation, not employment. It is important to emphasize, 
however, that despite gains in income following substance abuse treatment, earnings among the 
AFDC clients in the sample remained quite low-too low for most clients to achieve economic 
self-sufficiency. Unless these welfare clients receive vocational services in conjunction with 
substance abuse treatment, they will probably be unable to become economically self-sufficient. 

2. The Utility of Computerized State Administrative Databases for Tracking Client 
Outcomes and Performing Evaluation Research 

Few States have the resources or time to conduct studies involving large-scale primary data 
collection. Such studies may take years to complete and may produce information of diminished 
value given the rapid policy and programmatic changes taking place in many States. 

This study in Washington State illustrates a useful approach to performing evaluation research- 
that is, analyzing data provided by computerized State administrative databases. State databases 
contain client-level data representing important outcome indicators of substance abuse treatment- 
including employment, public assistance payments, Medicaid utilization and expenditures, and 
criminal justice information. By utilizing such information, States can greatly improve their 
capacity to perform timely evaluation and outcome studies. (Refer Chapter IV for details) 

Chapter II - Methods, Data, and Analytic Procedures 

a. Sample Selection Criteria and Description of the Study Population 

b. Treatment Modalities Analyzed and Assignment of Clients to Treatment and Comparison 

groups 

c. Data Sources and Construction of Analytic Files 


d. Measures 


e. Descriptive and Statistical Analytical Procedures 

A. Sample Selection Criteria and Description of the Study Population 

The study sample consisted of 5,664 individuals who (1) were admitted to substance abuse 
treatment in Washington State from July 1994 to June 1996; and (2) were receiving financial 










assistance from AFDC at the time of treatment admission. 

The client treatment database maintained by the Washington State Division of Alcohol and 
Substance Abuse was used to identify all persons admitted to substance abuse treatment from 
July 1994 through June 1996. Substance abuse treatment units serving publicly funded (Federal 
or State) clients in Washington State are required to report to this database, and approximately 
half of all treatment programs do so. Once individuals admitted to substance abuse treatment 
from July 1994 to June 1996 were identified, the database was searched again to identify 
individuals who were receiving financial support from AFDC at time of their admission to 
treatment. 


This two-step procedure resulted in the initial identification of 6,140 individuals in Washington 
State, admitted to substance abuse treatment from July 1994 through June 1996 and receiving 
financial assistance from AFDC at the time of their admission. Subsequently, 476 (7.7 percent) 
of these 6,140 clients were dropped from the study because either (1) they had a dual diagnosis 
(mental illness and substance abuse); or (2) data on key variables (e.g., discharge date or 
treatment modality) were missing for the individuals. 


The final study population, therefore, included 5,664 individuals admitted to substance abuse 
treatment from July 1994 through June 1996 who were receiving financial aid from AFDC at the 
time of their admission. Selected characteristics of the study population are shown in Table II-1. 



Table II-1. Selected Characteristics of the AFDC 

Study Population in Washington State (N = 5,664)* 

Characteristics 

Percent of total 
study population or 
mean 

(Standard 

deviation) 





Children in household 



0-1 

55.6% 


2-3 

37.9% 


$4 

6.4% 


Self-reported household 
monthly income 
(including AFDC income) 


$474 

($366) 


























Age at admission 

31.1 

(7.0) 


Race 



White 

72.2% 


Black 

12.3% 


Native American 

8.1% 


Asian and other (includes a 
small percentage of Hispanic 
clients) 

7.3% 


Female 

75.7% 


Single, divorced, widowed, or 
separated 

79.7% 


Education 



No high school degree 

37.5% 


High school degree only 

52.7% 


Vocational training or post- 
high school education 

7.7% 


Other 

2.1% 


Primary substance of abuse 



Heroin 

11.2% 


Cocaine 

15.7% 


Methamphetamines 

12.2% 


Alcohol 

48.6% 


Other drugs 

12.2% 


Frequency of drug/alcohol 

use 



Daily use 

31.9% 
































































Weekly use 

35.4% 


Less than weekly use 

32.6% 



*A11 of the 5,664 AFDC clients in the sample were admitted to publicly funded substance abuse 
treatment in Washington State from July 1994 through June 1996 and were receiving financial 
assistance from AFDC at the time of their admission to treatment. 

About 38 percent of the AFDC client households had two to three children, and 6.4 percent had 
four or more children. Slightly over half (56 percent) of the clients had one or no children 
(pregnant clients who meet income eligibility requirements qualify for AFDC). The self-reported 
monthly household income of AFDC clients averaged $474. Approximately 72 percent of the 
AFDC clients were white, 12 percent were black, 8 percent were Native American, and 7 percent 
were Asian, Hispanic, or other. More than one-third (38 percent) had no high school degree. 
Almost half reported alcohol as their primary substance of abuse; the others reported heroin, 
cocaine, methamphetamines or other drugs as their primary substance of abuse. About two-thirds 
reported abusing their primary substance on a daily or weekly basis. 

B. Treatment Modalities Analyzed and Assignment of Clients to Treatment and 
Comparison Groups 

The 5,664 AFDC recipients in the study population were admitted to a range of substance abuse 
treatment modalities, reflecting their different treatment needs (see Table II-2). 


Table II-2. Admissions of AFDC Clients to Substance Abuse 

Treatment by Treatment Modality at Index Admission* (N = 5,664) 

Treatment modality 

Number of 
admissions 

Percent 

Outpatient treatment 

3,784 

66.7% 

Intensive inpatient (21- or 28-day 
non-hospital) treatment 

706 

12.5% 

Detoxification 

626 

11.1% 

Methadone maintenance 

299 

5.3% 

Long-term residential treatment 

131 

2.3% 

Extended care 

60 

1.1% 

Recovery house 

58 

1.0% 

















































Total 


5,664 


100 . 0 % 


*A client's initial admission to substance abuse treatment is referred to as the index admission.As 
shown, about two-thirds (3,784) of the study population were initially admitted to an outpatient 
substance treatment program. Although all 5,664 AFDC clients were admitted to treatment at 
least once from July 1994 to June 1996, nearly half (46 percent) of the clients had more than one 
admission to treatment during this period. Specifically, 24 percent (1,369) had two admissions, 

12 percent (675) had three admissions, 6 percent (315) had four admissions, and the remaining 4 
percent (224) had more than four admissions. Overall, study population clients accounted for 
10,417 separate admissions to substance abuse treatment during the study period. 


Three substance abuse treatment modalities were selected for in-depth analysis: (1) outpatient 
treatment; (2) intensive inpatient (21- or 28-day non-hospital) treatment; and (3) methadone 
maintenance. Complicating the process of selecting specific treatment episodes for analysis, 
some clients in the study population were admitted to different modalities at different times. For 
example, a client could have an initial admission to outpatient treatment and a 
subsequentadmission to methadone maintenance. To address this situation, a series of decision 
rules was established to define treatment episodes for specific modalities. 

• Outpatient treatment episodes. Treatment episodes for outpatient treatment were limited 
to clients who received outpatient treatment at their index (first) admission. This rule 
ensured that the analysis focused on outpatient treatment provided as the primary 
modality, excluding outpatient treatment provided as followup to earlier inpatient 
treatment. 

• Intensive inpatient and methadone maintenance treatment episodes. For intensive 
inpatient treatment and methadone maintenance, treatment episodes were selected for 
analysis regardless of whether they represented the index admission. Thus, for example, a 
client receiving detoxification initially who was later admitted to intensive inpatient 
treatment would be classified as an inpatient client. 

AFDC clients were assigned to treatment and comparison groups for each of the three substance 
abuse treatment modalities being considered. Because all of 5,664 AFDC clients in the study 
population were admitted to substance abuse treatment, it was not possible to identify a "pure" 
comparison group that had no treatment. Comparison groups, therefore, consisted of clients who 
received minimal treatment or clients who received detoxification but no other treatment. For 
each modality of treatment, the treatment groups included clients who either completed treatment 
or stayed in treatment for substantial time. 

• Treatment and comparison groups for outpatient treatment. Clients staying in outpatient 
treatment a minimum of 90 days during their index admission formed the treatment 
group. Clients remaining in outpatient treatment fewer than 21 days during their index 
admission formed the comparison group. 




• Treatment and comparison groups for intensive inpatient (21- or 28-day non-hospital) 
treatment. Clients whose inpatient discharge record indicated "treatment completed" for 
either a 21- or 28-day treatment regimen formed the treatment group. It was not feasible 
to construct a comparison group for the inpatient analysis consisting of clients with 
minimal inpatient treatment. Instead, detoxification clients who had three or fewer 
detoxification treatments and no other treatment were used for the comparison group. 

• Treatment and comparison groups for methadone treatment. Methadone maintenance 
clients staying in treatment for a minimum of 120 days formed the methadone 
maintenance treatment group. Methadone clients staying in treatment fewer than 60 days 
formed the comparison group. 

The numbers of AFDC clients in the substance abuse treatment and comparison groups for each 
treatment modality are shown in Table II-3 below. On average, AFDC clients in the outpatient 
treatment group stayed in treatment 148 days; their counterparts in the outpatient comparison 
group stayed 9 days. AFDC clients in the methadone maintenance treatment group stayed 
intreatment for 262 days; their counterparts in the methadone maintenance comparison group 
stayed 29 days. 



Table II-3. Distribution of AFDC Clients 

Across the Three Substance Abuse Treatment Modalities (N = 5,664) 


Outpatient treatment: 

(n = 1,131) 


Intensive inpatient (21- or 28-day non-hospital) treatment: 


Methadone maintenance: 


C. Data Sources and Construction of Analytic Files 


Data for the 5,664 AFDC clients treated for substance abuse from July 1994 through June 1996 
were obtained from three computer databases maintained by Washington State agencies: 

• Data for clients receiving substance abuse treatment. The Washington State Division of 
Alcohol and Substance Abuse client database has information on the characteristics of 
individuals receiving substance abuse treatment and on their admission to treatment, 
discharge, and treatment activity. It includes data on age, gender, race, education, primary 
substance of abuse, frequency of use, past mental health treatment, probationary status, 
referral source, living arrangement, number of children in household, marital status, 
disability status, general health status, types of crimes committed, and household income. 




Data for employment and earnings. A database maintained by the Washington State 
Department of Employment Security includes wage data submitted by employers in the 
State each quarter. 


• Data for welfare payments. A database maintained by the Economic Services 

Administration within the Washington State Department of Social and Health Services 
has data on monthly cash amounts distributed to welfare recipients in the State. 

The Division of Alcohol and Substance Abuse client database was used to obtain activity data 
for individuals in outpatient or methadone treatment, including date and type of service provided. 
The Department of Employment Security database was used to obtain data on quarterly earnings 
for each of the AFDC clients in the study for the period from July 1993 through September 1998. 
The Economic Services Administration database was used to obtain data on the monthly cash 
grant amounts paid to each of the AFDC clients in the study over the 48-month period from July 
1993 to June 1997. 


These data from the three State administrative databases were entered into separate Access 
databases and then uploaded into SPSS data fdes. The individual SPSS data fdes were then 
combined into a master file that contained client, treatment, employment, and welfare payment 
data. The master file used for this study contained 10,417 records representing each treatment 
episode, or admission, for the 5,664 AFDC clients in the study population. 

D. Measuresl. Dependent VariablesKey dependent study measures were employment and 
welfare payment measures constructed using data from the Washington State administrative 
databases described above. The monthly welfare payment data were aggregated to reflect 
quarterly data to facilitate the analysis. 

• Employment outcome measures. Employment outcome data covering nearly 5 years (July 
1993 to September 1998) allowed the construction of outcome measures representing 8 
quarters of post-treatment followup. Three employment outcome variables were 
examined: (1) a binary variable set equal to 1 if a client had any positive earnings (> $0) 
during the 8 quarters following treatment and 0 otherwise; (2) a binary variable set equal 
to 1 if a client had earnings of at least $1,500 or more in at least 1 of the 8 quarters 
following treatment (the approximate equivalent of having a half-time job for a quarter at 
an hourly rate of $6) and 0 otherwise; and (3) total aggregate earnings in the 8 quarters 
following treatment, which reflected the combined effects of the number of hours worked 
and the hourly rate of pay. 

• Welfare payment outcome measures. Welfare payment outcome measures for clients in 
the study included quarterly welfare payment amount and number of quarters during 
which welfare payments were received. Welfare payment data covered a shorter time 
period (July 1993 to June 1997) than the employment data, necessitating a followup 
period for welfare payments of only 6 quarters. Consequently, the welfare payment 
analysis was limited to AFDC clients (n = 4,320) discharged from substance abuse 
treatment by January 1996. 



Information on selected employment and welfare payment measures for AFDC clients in the 
study population is shown in Table II- 4. 



Table II-4. Descriptive Information on Selected Dependent Measures for 
AFDC Clients 

Dependent measure 

Mean 

or 

percent 

Standard 

deviation 

Range 

Employment measures 

Aggregate earnings in 8-quarter 
followup period 
(n = 5,664) 

$4,811 

$9,581 

$0-$83,251 

Any employment (earnings > $0) 
in 8-quarter followup period 
(n = 5,664) 

58% 



Employment with earnings > 
$1,500 in at least 1 quarter of 8- 
quarter followup period 

39% 



Welfare measures 

Quarterly AFDC payments in 
6-quarter followup period 
(n = 4,320) 

$911 

$633 

$0 - $3,900 

Number of quarters (out of 6 
possible) with AFDC payments 
> $0 in followup period 
(n = 4,320) 

3.1 

2.6 

0-6 


2. Independent Variables 

Because assignment to treatment and comparison groups was not random, differences between 
the two groups of AFDC clients were controlled through statistical analysis. Independent client 
variables included (1) demographic variables (age, gender, race, years of education, marital 
status, number of children in household); (2) substance abuse variables (primary substance of 
abuse; frequency of use, severity of dependence, mode of drug intake-needle use versus other 
modes, number of previous admissions to substance abuse treatment before July 1994, number of 
admissions to substance abuse treatment from July 1994 to June 1996, and quarter of discharge), 
(3) general health variables (disabled, past hospitalization for mental illness, under current 





































treatment for mental health problem, and pregnant); and (4) employment and earnings variables 
(unemployed but seeking employment at time of admission, and baseline earnings-average 
quarterly earnings during the 4 quarters before substance abuse treatment). 

E. Descriptive and Statistical Analytical Procedures 

Both descriptive and multivariate statistical analyses of the data were perfonned, as described 
below. 

1. Descriptive Analysis 

The descriptive analysis identified patterns and trends in the 5,664 AFDC clients' earnings and 
welfare payments over time, both before and after substance abuse treatment. It also examined 
the pattern of employment and earnings among welfare recipients in the first year following 
TANF's implementation in Washington State 

2. Statistical Analysis 

The statistical analysis detennined the effects of substance abuse treatment on AFDC clients’ 
employment outcomes, making statistical adjustments for client characteristics (age, sex, race, 
education, primary substance of abuse, frequency of use, disability status, past mental health 
treatment, marital status, number of children in household, number of admissions prior to July 
1994, number of treatment episodes during the study period, and baseline earnings) that might 
affect these outcomes. A significance level of p = .05 was used for the analysis based on one-tail 
tests. Earnings and welfare payment data analyzed represented nominal dollars. 

In the statistical analysis, both logistic and multiple linear regression models were used: 

• Logistic regression was used to evaluate employment measures for AFDC clients in 
binary fonn (e.g., any post-treatment employment versus no employment). This type of 
regression model provides infonnation on the odds (likelihood) of the occurrence of an 
event in relation to some independent variable of interest. For example, in the context of 
the current study, there is interest in knowing whether AFDC clients who stayed in 
outpatient treatment for more than 90 days were more likely to become employed than 
clients who stayed less than 21 days. Logistic regression provides information to address 
this question. But it is also important to know whether treatment had some effect on the 
level of earnings for clients who became employed. 

• Ordinary least squares (OLS) regression was used to detennine the effects of treatment 
on the amount of earnings for clients who became employed at least part time. Because 
the earnings data exhibited some skewness, the OLS regressions were re-estimated using 
log-transformed data. The estimates were unchanged, so the original results (based on the 
untransfonned data) are reported. 

A separate subanalysis was performed to determine the effects of substance abuse treatment on 
post-treatment earnings among AFDC clients who had been on welfare for different lengths of 



time during the pretreatment period. Because of a larger number of cases were required, only 
outpatient clients were analyzed. In this analysis, the pretreatment baseline period was extended 
from 4 to 6 quarters by limiting the period of admission to substance abuse treatment to 1 Vi 
years (January 1995 to June 1996). Although this approach reduced the number of AFDC clients 
that could be analyzed, it made it possible to create a better stratification of pretreatment time on 
welfare. 


For the subanalysis, AFDC outpatient clients in the study population were divided into two 
groups depending on the number of quarters they received welfare payments during the 6-quarter 
baseline period. Clients who had received welfare payments for 4 or more quarters were included 
in one group, and clients who had received welfare payments for 3 or fewer quarters were 
included the second group. A binary variable representing the two groups was created, and this 
variable was then interacted with the treatment variable. Including this interactive tenn in the 
regression models made it possible to examine the effects of treatment for clients with different 
levels of time on welfare. 

Chapter III - Results: The Impact of Substance Abuse Treatment on Employment Outcomes 
Among AFDC Clients 

This chapter presents findings from the descriptive and statistical analysis regarding the effects 
of substance abuse treatment on employment outcomes for 5,664 AFDC clients who were 
admitted to publicly funded substance abuse treatment in Washington State. 

a. AFDC Clients' Earnings and Welfare Payments Before and After Substance Abuse Treatment 

b. Statistical Analysis 

c. Descriptive Analysis 

A. AFDC Clients' Earnings and Welfare Payments Before and After Substance Abuse 
Treatment: 

Trends and patterns in earnings and welfare payments for the study's 5,664 AFDC clients over 
time are noted below. Because the data are not adjusted to account for differences in age, gender, 
education, or other factors that may affect treatment outcomes, conclusions about the effects of 
substance abuse treatment on AFDC clients' earnings and welfare payments should not be made 
on the basis of this information alone. Results of the multivariate statistical analysis, controlling 
for independent variables, are presented in the section that follows. 

1. Profile of AFDC Clients' Earnings Before and After Substance Abuse Treatment 

As shown in Figure III-1, quarterly earnings for the AFDC clients in the study population were 
modest both before and after substance abuse treatment. Average quarterly earnings for all 5,664 
clients as well as for the 3,884 clients who had any earnings, initially declined before substance 
abuse treatment and then rose after treatment. 





The bottom line in Figure III-1 indicates that average quarterly earnings for the population of all 
5,664 AFDC clients rose every quarter after substance abuse treatment. Even at their maximum, 
however, average quarterly earnings for the 5,664 clients were only $800, or just $267 per 
month. 

Among the 3,884 AFDC clients with any earnings whatsoever (earnings > $0) in the tracking 
period, average quarterly earnings increased by approximately 30 to 40 percent after treatment, 
as shown by the upper line in Figure III-1. Even for this subgroup of AFDC clients, though, 
average quarterly earnings were still very modest 8 quarters after substance abuse treatment-on 
the order of $1,200, or $400 per month. 



Figure III-l. AFDC Clients Quarterly Earnings Before 
and After Substance Abuse Treatment 
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Figure III-2. AFDC Clients’ Quarterly Earnings Before 
and After Substance Abuse Treatment 
by Education Level 
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Average quarterly earnings among the 5,664 AFDC clients varied by education level, as 
illustrated in Figure III-2. Before substance abuse treatment, the clients with a high school degree 
or post-high school training earned more than clients without a high school degree. The 
differential grew 
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Figure III-3. AFDC Clients’ Quarterly Earnings Before and After 
Substance Abuse Treatment by Racial/Ethnic Group 
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after substance abuse treatment. By 8 quarters after treatment, AFDC clients with a high school 
degree earned 35 percent more than the clients without a high school degree. Earnings also 
varied by racial/ethnic group. Both before and after substance abuse treatment, average quarterly 
earnings were higher for some racial/ethnic groups than for others. As shown in Figure III-3, 
Asian and other clients (including a small percentage of Hispanics) had the highest average 
quarterly earnings; white clients had the second highest average quarterly earnings; and black 
and Native American clients had the lowest average quarterly earnings. 

The distribution of aggregate income levels among the 5,664 AFDC clients over the 8-quarter 
period following substance abuse treatment is shown in Table III-1. Forty-two percent of the 
clients had no earned income; 14 percent earned between $1 and $1,000; and another 11 percent 
earned between $1,001 and $3,000. Only 11 percent of the 5,664 AFDC clients had aggregate 
earned income exceeding $15,000 during the 8-quarter period following treatment. 


Table III-l. Distribution of Aggregate Income Levels 
Among AFDC Clients in the 8 Quarters 
Following Substance Abuse Treatment (N = 5,664) 


Income Levels 

Percent of AFDC Clients 

$0 (no earnings) 

42% 

$1 to $1,000 

14% 

$1,001 to $3,000 

11 % 



































$3,001 to $8,000 

13% 

$8,000 to $15,000 

9% 

$15,001 + 

11 % 


Figures III-4a and III-4b present employment and earnings data for substance abuse treatment 
and comparison groups for each of the three treatment modalities. The percentage of AFDC 
clients obtaining at least part-time employment (indicated by positive earnings in the 8-quarter 
followup period) was greater among AFDC clients in treatment groups than among clients in 
comparison groups (Figure III-4a). Almost 66 percent of AFDC clients in the intensive inpatient 
treatment group had earnings, compared with 49 percent of clients in the inpatient comparison 
group. Employment status differences between treatment and comparison groups for the other 
modalities (outpatient and methadone maintenance) were smaller. 

Aggregate earnings among the AFDC clients obtaining at least part-time employment were 
substantially greater for treatment group clients than for comparison group clients (Figure III- 
4b). For example, the average aggregate income for clients in the outpatient treatment group was 
$10,718, compared with $6,954 for clients in the outpatient comparison group. 


Figure III-4a. Percentage of AFDC Clients With Positive 
Earnings in the 8-Quarter Followup Period by 
Treatment Modality: Treatment and Comparison Groups 
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Figure III-4b. Aggregate Earnings Among AFDC Clients 
Obtaining at Least Part-Time Employment* During the S-Quarter 
Followup Period by Treatment Modality: 

Treatment and Comparison Groups 
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♦Pait-tme employmentwas defined as haring earned income duringthe followup period. 


2. Profile of Welfare Payments to AFDC Clients Before and After Substance Abuse 
Treatment 

The analysis of welfare payments was restricted to the 4,320 AFDC clients discharged from 
substance abuse treatment in Washington State by January 1996. Since welfare payment data 
were available only through June 1997, clients discharged after January 1996 would have had 
fewer than 6 quarters of welfare data available. 

Average Quarterly Welfare Payment Amounts to AFDC Clients. Figure III-5 shows the actual 
welfare payment amounts per quarter over the 10-quarter tracking period for the 4,320 AFDC 
clients. Quarterly welfare payments to these clients increased from $848 to $976 during the 4- 
quarter period before substance abuse treatment, reached $1,079 in the first quarter after 
treatment, then gradually declined to $762 in the 6 quarters after treatment. 

At the time of this study, the average AFDC payment in Washington State was $349 per month 
for a two-person family (one parent and one child) and $440 per month for a three-person family. 
Beneficiaries could earn up to double their monthly welfare payment and still remain eligible for 
cash grants, food stamps, and medical assistance, but their cash payment would be reduced, 
depending upon the level of income they reported. 




Figure m-5. AFDC Clients’ Quarterly Welfare Payments 
Before and After Substance Abuse Treatment (n = 4,320) 
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Number of Quarters That AFDC Clients Spent on Welfare. The 4,320 AFDC clients varied with 
respect to the amount of time they spent on welfare, as shown in Figure III-6. During the 10- 
quarter tracking period (4 quarters before treatment and 6 quarters after treatment), about 14 
percent of the clients received cash grants for only 1 or 2 quarters, 30 percent received payments 
for 3 to 6 quarters, nearly 30 percent received payments for 7 to 9 quarters; and 27 percent 
remained on welfare continuously. 





Figure DI-6. Length of Time on Welfare Among AFDC 
Clients: Number of Quarters out of 10 
Quarters That Clients Received 
Cash Grants (n = 4,320) 



Trends in Quarterly Welfare Payments to AFDC Clients in the 6 Quarters Following Substance 
Abuse Treatment. The three figures below show trends in quarterly welfare payments during the 
6-quarter period after substance abuse treatment for AFDC clients in treatment and comparison 
groups for outpatient treatment (Figure III-7), intensive (non-hospital) inpatient treatment 
(Figure III-8), and methadone maintenance (Figure III-9). 



















Figure DI-9. Quarterly Welfare Payments to AFDC 
Clients Following Methadone Treatment: Treament and 
Comp ails on Groups 
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Welfare payments to AFDC clients in the treatment groups declined by 25 to 35 percent in the 6 
quarters following substance abuse treatment. For clients in the outpatient treatment group, 
quarterly welfare payments declined by 31 percent in the 6-quarter followup period, from $1,130 
per quarter to $785 per quarter. For clients in the inpatient treatment and methadone groups, the 
respective reductions were 27 percent and 25 percent. The pattern of declining welfare payments 
was similar for comparison group clients. 

This pattern of reduced welfare payments to AFDC clients in the 6-quarter post-treatment period 
is consistent with the pattern of increased post-treatment earnings shown in Figure III-1. 
Statistical analysis showed that post-treatment earnings and welfare payments were negatively 
correlated (r= -.24, p < .01). It is important to note however, that factors besides income affected 
clients' welfare payments. First, AFDC clients in Washington State were subject to certain work 
requirements during the study period; failure to meet these requirements could result in a 
sanction and suspension of welfare payments. Second, a change in AFDC clients' living 
arrangement or housing situation could also affect the clients' cash grants. Third, some AFDC 
clients may have moved out of the State or died. 

Information on these conditions and changes was unavailable at the time of this study. Given 
their existence, however, readers should use caution in interpreting the welfare payment data 
presented in this report. It was not possible to control for these unmeasured factors, so assessing 
the effects of treatment on welfare payments through formal statistical analysis was viewed as 
inappropriate. 



B. Statistical Analysis: 


The Effects of Substance Abuse Treatment on AFDC Clients' Employment and Earnings 

Statistical techniques (logistic and multiple linear regression) were used to detennine the effects 
of substance abuse treatment on employment outcomes among the AFDC clients adjusting for 
various client characteristics identified previously. The major findings of the statistical analysis 
are presented below. More detailed results are presented in the technical appendix at the end of 
this document. 

1. Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes 


Regression analysis, using two different measures of employment outcomes, indicated that 
substance abuse treatment had a positive effect on employment outcomes for AFDC clients in 
the study population. 

In the first logistic regression analysis, employment was defined as any earned income 
whatsoever. Figure III-10 shows the results, presented as adjusted odds ratios that AFDC clients 
who received substance abuse treatment would have any earned income during the 8-quarter 
period following substance abuse treatment. By definition, a comparison group has an odds ratio 
of 1.00. Thus, an odds ratio of 1.64 for a treatment group indicates that the average client in the 
treatment group is 64 percent more likely than a counterpart in the comparison group to have 
earned income, other factors held constant. Although only one comparison group is illustrated in 
the figure, the odds ratios presented are based upon analyses of separate comparison groups 
corresponding to the three treatment groups (see Appendix). 


Figure IH-10. Adjusted Odds Ratios of AFDC Clients Having 
Any Earned Income During the 8-Quarter Followup Period by 
Treatment Modality: Treatment and Comparison Groups 
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As shown in Figure III-10, AFDC clients in each of three substance abuse treatment groups- 
intensive inpatient, outpatient, and methadone maintenance-were more likely than their 
counterparts in the comparison groups to have some earned income in the 8-quarter period 
following substance abuse treatment. Clients completing intensive inpatient treatment were 64 
percent more likely to have earned income than their comparison group counterparts. Clients in 
the outpatient treatment group (clients staying in outpatient treatment for 90 days or more) were 
26 percent more likely to have earned income than their comparison group counterparts (clients 
staying in outpatient treatment less than 21 days). Clients in the methadone treatment group 
(clients completing at least 120 days of methadone treatment) were 50 percent more likely to 
have earned income than their comparison group counterparts (clients staying in methadone 
treatment less than 60 days). Each of the odds ratios shown in the figure achieved statistical 
significance (p < .05). 

Figure III-11 shows the results of a second logistic regression analysis, with employment defined 
as earnings of at least $1,500 in 1 of the 8 followup quarters. (An income of $1,500 in a quarter 
is roughly equivalent to working half time at an hourly rate of $6.) The magnitude of the 
substance abuse treatment effect is somewhat larger when this measure of employment is used. 


Figure III-ll. Adjusted Odds Ratios of AFDC Clients Having Earned 
Income of $1,500 or More During the 8-Quarter Followup Period by 
Treatment Modality: Treatment and Comparison Groups 
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AFDC clients in the three substance abuse treatment groups were 60 to 100 percent more likely 
than their counterparts in the comparison groups to have earned at least $1,500 in 1 or more 
quarters in the 8-quarter followup period. Clients in the inpatient and outpatient treatment groups 
were about 65 percent more likely than their counterparts in comparison groups to obtain work 
and earn at least $1,500 in 1 of the followup quarters (p < .05). Clients in the methadone 
treatment group were twice as likely as their counterparts in the comparison group to have 
earned $1,500 in a quarter (p < .05). 





2. Effects of Substance Abuse Treatment on Aggregate Earnings Among AFDC Clients 
Who Became Employed at Least Part Time 

The 58 percent of AFDC clients in the study population who had at least some earnings in the 8 
quarters following substance abuse treatment (see Table III-1 presented earlier) were included in 
the earnings analysis. (Data on the actual number of hours worked by these individuals were 
unavailable.) 

An OLS regression analysis was performed to examine the impact of treatment on aggregate 
earnings for AFDC clients (see Appendix). The results are depicted in Figure III-12, which 
shows the adjusted aggregate earned income of the average AFDC client in treatment and 
comparison groups for each of the three treatment modalities. In this context, average means 
having characteristics representative of the majority of clients within a given treatment modality 
(i.e., being white, a woman, single, of average age and education, with a primary substance of 
abuse of alcohol, and having other representative characteristics). 




Figure 111-12. AFDC Clients' Adjusted Aggregate Earnings in the 
8-Quarter Followup Period by Treament Modality: 
Treatment and Comparison Groups 
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Aggregate earned income was higher for the average AFDC client in each of the three substance 
abuse treatment groups than for the average AFDC client in the comparison groups. Differences 
in earnings for clients in treatment and comparison groups ranged from $1,723 for clients in 
intensive inpatient treatment to $3,280 for clients in outpatient treatment (p-value range, .001 to 
.05). 

3. Effects of Time on Welfare on AFDC Clients' Post-Substance Abuse Treatment Earnings 

The question of whether AFDC clients' time on welfare prior to receiving substance abuse 
treatment affected post-treatment employment and earnings was examined through a subanalysis 





of AFDC clients in outpatient treatment (there were too few clients in the other two modalities to 
pennit analysis). Time on welfare was found to have an effect on post-treatment earnings. 

AFDC clients in the study population were divided into two groups, depending on the number of 
quarters they received welfare cash grants in the 6-quarter period prior to receiving substance 
abuse treatment. The first group included AFDC clients who had received welfare payments in 4 
or more of the 6 quarters (57 percent of clients); the second included clients who had received 
welfare payments for 3 or fewer of the 6 quarters (43 percent of clients). The analysis then 
compared aggregate earnings in the 2 years following treatment for clients in these two groups 
(see Appendix). 
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Figure III-13. Differences in Aggregate Earnings in the 8-Quarter 
Followup Period Among AFDC Outpatient Clients by Time on 
Welfare: Treatment and Comparison Groups 
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Figure III-13 shows that outpatient AFDC clients' time on welfare in the 6-quarter pre-treatment 
period had a substantial effect on their aggregate earnings. Individuals who had been on welfare 
for 4 or more quarters in the 6-quarter pretreatment period had lower aggregate earnings in the 2- 
year post-treatment period than individuals who had been on welfare for 1 to 3 quarters in the 6- 
quarter pretreatment period. Clients in the outpatient treatment group who had been on welfare 4 
to 6 quarters prior to treatment had predicted aggregate earnings of $3,993 over the 2-year post¬ 
treatment period; their comparison group counterparts had predicted aggregate earnings of only 
$1,615 over the same period (p < .05). 

Comparable and statistically significant differences were found for AFDC clients who had been 
on welfare for fewer than 4 quarters in the pretreatmentperiod. Clients in the treatment group 
who had been on welfare 1 to 3 quarters prior to treatment had predicted aggregate earnings of 




$6,192 over the 2 years following treatment; their comparison group counterparts had predicted 
aggregate earnings of $3,814 for the same period. 

C. Descriptive Analysis: 

AFDC Clients' Employment and Earnings in the First Year of TANF's Implementation. 

TANF was implemented in Washington State in the third quarter of 1997. Data from State 
administrative databases were used to perform a preliminary analysis of the pattern of 
employment and earnings among welfare recipients during the first year following 
implementation. For this analysis, no treatment/control-group comparisons were made; the data 
were treated as purely observational. 

Figure III-14 shows trends in the percentage of AFDC clients in the study population who 
became employed-using as the measure of employment earned income of at least $100 in a given 
quarter-over the period from 1993 to 1998. The percentage of AFDC clients with quarterly 
earned income of at least $100 increased over time, but this trend began well before Washington 
State's implementation of TANF in the third quarter of 1997. Nonetheless, the positive trend. 


Figure HI-14. Percentage of AFDC Clients With 
Quarterly Earned Income of $100 or More 
1993- 1998 
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continued such that by the third quarter of 1998, 38 percent of the 5,664 AFDC clients in the 
study population had earned income of at least $100. Figure III-15 shows trends in average 



quarterly earnings among AFDC clients in the study population-all 5,664 clients and the 2,793 
clients with quarterly earnings of at least $500-over a 5-year tracking period from 1995 to 1998 

For both groups of AFDC clients, there is a trend of increasing earnings, but again the trend 
began well before TANF's implementation in the third quarter of 1997. By the third quarter of 
1998, the average quarterly earned income among all 5,664 AFDC clients (about 60 percent of 
whom hadno income) was $1,004. Even among the 2,793 clients who participated in the labor 
force, as indicated by having earnings of at least $500, the average quarterly earned income in 
the third quarter of 1998 was a very modest amount-$ 1,689, or $563 per month. 


Figure III-15. AFDC Clients’ Quarterly Earnings 

1993- 1998 
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Chapter IV - Conclusions and Implications 

This study had two purposes. The primary purpose was to assess the effect of substance abuse 
treatment on employment and earnings among AFDC clients admitted to treatment in 
Washington State. A second purpose was to illustrate the potential utility of computerized State 
administrative databases for tracking client outcomes and conducting timely evaluation research. 




a. The Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes 


b. The Utility of Computerized State Administrative Databases for Tracking Client Outcomes 

and Performing Evaluation Research 

A. The Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes 

This study's finding that substance abuse treatment was associated with increased employment 
and earnings among AFDC clients during a 2-year post-treatment period is notable, especially in 
light of the fact that the primary goal of substance abuse treatment is rehabilitation, not 
employment. This finding was robust across the three treatment modalities examined: outpatient 
treatment, intensive inpatient (21- or 28-day non-hospital) treatment, and methadone 
maintenance. AFDC clients in these treatment groups were more likely than their counterparts in 
comparison groups to become employed-whether employment was measured in terms of any 
earnings whatsoever or quarterly earnings above $1,500. Among the subset of clients who were 
employed, those in treatment groups had aggregate income that exceeded by $1,700 to $3,200 
(30 to 60 percent) aggregate income for clients in comparison groups. 

Time on welfare proved to have a substantial effect on AFDC clients' earnings prospects. AFDC 
clients who had been on welfare for 4 to 6 quarters of the 6-quarter pretreatment period had 
lower aggregate earnings in the 8-quarter post-treatment period than clients who had been on 
welfare for only 1 to 3 quarters. Regardless of the amount of time AFDC clients had spent on 
welfare, however, outpatient substance abuse treatment was associated with a significant 
increase (p < .05) in earnings. Among AFDC clients who had been on welfare for 4 to 6 quarters 
in the pretreatment period, those who received outpatient substance abuse treatment had 
aggregate earnings of $3,993 in the 8 quarters following treatment; their counterparts in the 
outpatient comparison group earned only $1,615 in the same period. Among AFDC clients who 
had been on welfare for 1 to 3 quarters in the pretreatment period, those who received outpatient 
substance abuse treatment had aggregate earnings of $6,192 in the 8 quarters following 
treatment; their counterparts in the outpatient comparison group earned only $3,814 in the same 
period. 

Because this study did not randomly assign subjects to treatment and comparison groups, an 
important question is whether there is selection bias. Random assignment of subjects is 
alwayshighly desirable, but is not possible for studies that rely on secondary analysis of data 
obtained from administrative databases. Thus, appropriate caution must be used in the 
interpretation of the findings from this study (and similar studies). 

The validity of a study such as this rests largely on the extent to which statistical analysis is able 
to control for factors that may affect outcomes and treatment classification. Although the study 
was able to control for a large number of factors-including differences in AFDC clients' 
pretreatment earnings-unmeasured selection effects could still have influenced the results in 
some u nkn own way. 

The study's ability to control for differences in AFDC clients' pretreatment earnings, though not 
eliminating possible selection problems, does provide some measure of reassurance. The 





measure of AFDC clients' (pretreatment) baseline earnings was consistently the most important 
independent predictor of post-treatment earnings. Regression models indicated that for every $ 1 
of (pretreatment) baseline income, AFDC clients in this study earned approximately $2 to $4 in 
post-treatment income, other things being equal. In short, the AFDC clients who worked more 
before receiving treatment for substance abuse tended to work more after receiving such 
treatment. 

The study's findings regarding the positive effects of substance abuse treatment on AFDC clients' 
employment and earnings need to be balanced with the observation that earnings were relatively 
low among the AFDC clients in the sample-too low for the clients to achieve self-sufficiency. 
Among the 5,664 AFDC clients in the study population, 42 percent had no earned income in the 
8-quarter post-treatment period, and an additional 14 percent had aggregate earned income in 
that period of under $ 1,000. 

TANF has introduced strong incentives to work. Thus, AFDC clients today may be earning 
somewhat more than the data presented in this report suggest. But as recently as the third quarter 
of 1998-1 year after TANF's implementation-the level of quarterly earnings among the 
Washington State AFDC clients in the study population was very modest (see Figure III-15), 
despite the incentives for employment. 

Among individuals with substance abuse problems, the goals of employment and self-sufficiency 
underpinning TANF may be achievable for only a small minority unless ancillary vocational 
services are provided in conjunction with substance abuse treatment. Earlier studies from 
Washington State indicate that vocational services enhance employment and earnings when 
delivered in conjunction with treatment. Unless AFDC clients treated for substance abuse 
problems receive vocational services, there is little likelihood that many of them will be able to 
find jobs and become economically self-sufficient. There is a very real possibility that AFDC 
clients who are unable to obtain employment may be forced to rely on other local and State 
health and welfare systems once their time allotment for financial assistance under TANF is 
exhausted. If so, the costs of operating these health and welfare systems will certainly increase. 

B. The Utility of Computerized State Administrative Databases for Tracking Client 
Outcomes and Performing Evaluation Research 

State substance abuse agencies are under increasing pressure to demonstrate accountability for 
their budgets and must plan and administer substance abuse programs in the face of rapidly 
changing environments. These needs require the capability to perform timely analyses and 
outcome evaluations at a reasonable cost. 

State computer databases contain client-level data representing important outcome indicators of 
substance abuse treatment-including employment, public assistance payments, Medicaid 
utilization and expenditures, and criminal justice infonnation. This study in Washington State 
illustrates the potential utility of State administrative databases in tracking clients' outcomes and 
conducting timely evaluation research. 



The experience of Washington State in conducting outcome studies demonstrates the important 
advantages of establishing a database infrastructure that can provide data in a timely and efficient 
manner for evaluation and research purposes. The advantages of using State administrative 
databases include the following: 

• Clients can be tracked over extended periods of time. 

• Baseline (pretreatment) data can be incorporated into the analysis, reducing problems of 
selection bias. 

• Large numbers of clients can be studied at little extra cost. 

• Problems of client recall and case attrition are reduced. 

From the standpoint of perfonning evaluations, however, State databases do have inherent 
limitations. Since the information contained in computer databases is collected for administrative 
purposes, the desired measures may not be available in the preferred fonn. The types of 
information gathered and stored in State databases is subject to change, and changes may create 
problems for longitudinal analyses. Finally, information contained in State administrative 
databases may not be updated frequently enough to make the information as current as might be 
desired. 

A number of useful lessons emerged from this study that merit brief discussion: 

• First, it is important that a person familiar with the different State databases either extract 
the data or serve as a communication link with others who will perform this function. 
Many detailed questions arise regarding the definition of variables and the organization 
and completeness of the information contained in databases that an outside researcher 
cannot anticipate. Addressing these questions early-before data extract files are created- 
helps to avoid subsequent analytical problems. 

• Second, to the extent possible, data should be collected that will allow the triangulation of 
measures. As part of this study, detailed treatment activity data were collected and used 

to verily discharge-date information obtained from theclient discharge records. This 
proved very useful in improving the reliability of the discharge information. 

• Third, it is generally useful to obtain data on more cases than research design 
considerations (e.g., ensuring statistical power and external validity) may dictate. 
Inevitably some cases will have to be dropped because of incomplete or unreliable data. 
Further, the need for subanalyses that were not initially planned may emerge, which may 
require additional cases. 

The increasing complexity of the policy and program environment of substance abuse treatment 
is placing more emphasis on timely evaluation and outcomes research. By utilizing information 
available in computer databases, States can greatly improve their capacity to perform timely 
evaluation and outcome studies. 
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21. Technical Appendix 

22. This appendix provides additional information regarding the multivariate analyses 
presented in the body of the report. The findings of these analyses were presented in 
Figures 111-10 through 111-13 in Chapter III. Shown in Tables A-l through A-4 below are 
the estimated regression coefficients generated by each of the multivariate analyses, 
along with the 95 percent confidence intervals and p-values. 

23. Two type of multivariate analyses were conducted: (1) logistic regression, and (2) 
ordinary least squares (OLS) regression. To examine the effect of treatment on the 
likelihood of becoming employed, logistic regression was used. The general estimation 
model for the logistic regression is shown below: 

24. Logit Model : 

25. Log [P;■ ) (1 - P,)] = " + $iXn + $ 2 X 2 / + ,i where " is a constant term, Xi is a variable 
denoting some treatment group (outpatient, inpatient or methadone treatment), X 2 is a 
vector of control variables and , is an error term. Figures III-10 and III-11 show the 
adjusted odds ratios generated from the $1 logistic regression estimates. These estimates 
provide information about the likelihood of some employment outcome occurring 
holding other factors constant. The estimates generated are in the form of log odds and 
are exponentiated to obtain the estimated adjusted odds ratios (shown in the tables 
below). In addition to examining the effect of treatment on the likelihood of becoming 
employed, the study assessed the effect of treatment on earnings among those clients who 
became employed during the followup period (Figures III-12 and III-13). OLS regression 
was used to estimate the effect of treatment on the level of earnings. The general OLS 
model is shown below: 

26. OLS Model : 

27. Y, = " + $ 1 X 1 ,- + $ 2 X 2 / + „■ where " is a constant tenn, Xi is a variable denoting some 
treatment group (outpatient, inpatient or methadone treatment), X 2 is a vector of control 
variables and , is an error tenn. Figure III-12 shows the $1 OLS regression estimates. 
Figure III-13 presents the results of the interaction model constructed to estimate the 



effects of outpatient treatment for clients with differing amounts of time on AFDC prior 
to treatment. This model is the same as the model depicted above, except that it includes 
an interaction tenn. The estimated OLS coefficients indicate the difference in earnings 
between the treatment and comparison groups holding other factors constant. 

28. A. Logistic Regression Estimates 

29. Tables A-l and A-2 below show the estimated odds ratios depicted in Figures III-10 and 
III-11, respectively. The effects of other independent (control) variables and the statistical 
properties of the models are also described. 



Table A-l. Likelihood of Any Employment 

Treatment 

Group 

Adjusted Odds 
Ratio 

95% 

Confidence 

Interval 

P-Value 
(one-tailed test) 

Intensive 

Inpatient 

1.64 

1.17-2.29 

.001 

Outpatient 

1.26 

1.03 - 1.53 

.013 

Methadone 

1.50 

1.01 -2.27 

.05 


30. The -ILL values for the intensive inpatient, outpatient and methadone treatment logistic 
models were, respectively, 1614, 2478 and 467. The associated P 2 values were 229, 245 
and 57. The significance levels for the models overall were < .001. There was no clear 
pattern of findings with regard to the effects of other variables on the odds of achieving 
any employment. 

31. The inpatient and outpatient treatment models had greater statistical power than the 
methadone model due to the greater numbers of observations. In the inpatient model, 
years of education, baseline earnings, Asian ethnicity and male gender were positively 
associated (p < .05) with the odds of achieving employment in the followup period. 
Native Americans were less likely to obtain employment compared to whites. Clients 
who used needles as the mode of drug taking and clients receiving mental health 
treatment at time of treatment intake were also less likely (p < .05) to obtain employment. 

32. In the outpatient model, older clients, Native Americans, clients with chronic disease and 
clients with previous outpatient treatment in the past 12 months were less likely (p < .05) 
to become employed during the followup period. Baseline earnings were a strong 
(positive) predictor of post-treatment employment. Asian clients were more likely than 
white clients to achieve employment. 

The adjusted odds ratios, along with 95 percent confidence intervals and p-values, for 
Figure III-11 are presented in Table A-2. 








Table A-2. Likelihood of Earning $1,500 or More in 1 Out of 8 Quarters 

Treatment 

Group 

Adjusted Odds 
Ratio 

95% 

Confidence 

Interval 

P-Value 
(one-tailed test) 

Intensive 

Inpatient 

1.67 

1.17-2.29 

.005 

Outpatient 

1.64 

1.37-1.97 

<.001 

Methadone 

2.01 

1.53-2.69 

.005 


33. 

The -ILL values for the intensive inpatient, outpatient and methadone treatment logistic 
models were, respectively, 1519, 2273 and 379. The associated P values were 219, 335 
and 49. The significance levels for the models overall were < .005. The same general 
pattern of effects as discussed above for the initial logistic regression analysis was 
observed for this analysis. Compared to white clients, Native American clients were less 
likely to achieve earnings of $1,500 or more in 1 of the 8 followup quarters, while Asian 
clients were more likely than white clients to obtain this level of employment. Higher 
baseline earnings and male gender were associatedwith a greater likelihood of achieving 
employment at the specified earnings level. Outpatient and inpatient clients who used 
needles as their mode of drug taking were less likely (p < .05) to achieve employment 
than clients who did not use needles. 

34. B. OLS Regression Estimates 

35. Tables A-3 and A-4 below show the OLS results depicted in Figures III-12 and III-13, 
respectively. 



Table A-3. Adjusted Aggregate Earnings in the 8-Quarter Followup 

Period 

Treatment 

Group 

Estimated 

Coefficient 

95% 

Confidence 

Interval 

P-Value 
(one-tailed test) 

Intensive 

Inpatient 

$1,723 

$18-$3,428 

.045 

Outpatient 

$3,280 

$2,215 -$4,345 

<.001 

Methadone 

$2,929 

$12-$5,846 

.048 
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36. The F statistics and adjusted R for the three OLS models summarized in Table A-3 were 
as follows: Intensive Inpatient Treatment-F = 20.8 ( 22 , 854 ), adjusted R 2 = .33; Outpatient 
Treatment-F = 24.0 (22,1,147), adjusted R 2 = .30; and Methadone Treatment-F = 2.2 ( i 6 ,i 50 ), 
adjusted R 2 = .11. Among the control variables, baseline earnings was the strongest and 
most consistent predictor of post-treatment earnings. For every $1 in the pretreatment 
baseline period, clients earned from approximately $1.65 to $4.60 more in the followup 
period, other things equal. Years of education was also a strong and consistent predictor 
of post-treatment earnings. Other than these two variables there were no other control 
variables consistently associated with post-treatment earnings. In the inpatient treatment 
equation, needle use was associated (p = .03) with lower earnings. Variables representing 
race/ethnic group were not related to post-treatment earnings.The explanatory power of 
the outpatient treatment interaction model (Table A-4 and Figure III-13) was modest: F = 
4.6(24,1282), R“ = .09. The estimated treatment, time-on-welfare, and interaction 
coefficients are shown in the Table A-4. 



Table A-4. Differences in Post-Treatment Aggregate Earnings Among 

Outpatient 

Clients Having Different Amounts of Time on Welfare 

Measure 

Estimated 

Coefficient 

95% 

Confidence 

Interval 

P-Value 
(one-tailed test) 

Outpatient 

Treatment 

$2,378 

$18-$3,428 

.004 

Time on 

Welfare 

-$968 

-$1,918--$16 

.041 

Interaction 

Tenn 

-$1,232 

-$2,442 - -$22 

.048 


37. Few control variables were found to be associated with post-treatment earnings in the 
interaction model. Male gender was positively associated with earnings, as was years of 
education. Daily use of alcohol/drugs and previous mental health treatment were 
negatively associated with earnings. 































